קורס גרפיקה ממוחשבת 2008 סמסטר ב' Image Processing 1 חלק מהשקפים מעובדים משקפים של פרדו דוראנד, טומס פנקהאוסר ודניאל כהן-אור

Transcription

1 קורס גרפיקה ממוחשבת 2008 סמסטר ב' Image Processing 1 חלק מהשקפים מעובדים משקפים של פרדו דוראנד, טומס פנקהאוסר ודניאל כהן-אור

2 What is an image? An image is a discrete array of samples representing a continuous 2D function Continuous function Discrete samples 2

3 Amplitude Converting to digital form Convert continuous sensed data into digital form Quantization Sampling 3

4 Sampling and Reconstruction Sampling Reconstruction

5 Sampling and Reconstruction Figure 19.9 FvDFH

6 Sampling Theory How many samples are required to represent a given signal without loss of information? What signals can be reconstructed without loss for a given sampling rate?

7 What happens when we use too few samples? Aliasing Aliasing Figure FvDFH

8 Spectral Analysis So our image (function f(x,y)) describes how the signal changes over time (x and y axes) Aliasing occurs when we use too few samples (what is enough?) The more an image changes, the more we need to sample it. How do we measure how fast a signal changes? Frequencies 8

9 Spectral Analysis Spatial domain: Function: f(x) Filtering: convolution Frequency domain: Function: F(u) Filtering: multiplication Any signal can be written as a sum of periodic functions.

10 Fourier Joseph Fourier discovered in 1822 that Any periodic function can be expressed as the sum of sines and/or cosines if different frequencies (Fourier Series) Even functions that are not periodic can be expressed as the integral of sines and/or cosines (Fourier Transform) Initial application was in heat diffusion 10

11 Fourier Transform (1D) Figure 2.6 Wolberg

12 Fourier transform: F( u) Fourier Transform (1D) f ( x) e Inverse Fourier transform: i2xu dx f ( x) F ( u) e i2ux du

13 Sampling Theorem A signal can be reconstructed from its samples, if the original signal has no frequencies above 1/2 the sampling frequency - Shannon The minimum sampling rate for bandlimited function is called Nyquist rate A signal is bandlimited if its highest frequency is bounded. The frequency is called the bandwidth.

14 Image Processing Pixel operations Add random noise Add luminance Add contrast Add saturation Filtering Blur Detect edges Sharpen Emboss Median Quantization Uniform Quantization Floyd-Steinberg dither Warping Scale Rotate Warps Combining Composite Morph

15 Adjusting Brightness Simply scale pixel components Must clamp to range (e.g., 0 to 1) Original Brighter

16 Adjusting Contrast Compute mean luminance L for all pixels luminance = 0.30*r *g *b Scale deviation from L for each pixel component Must clamp to range (e.g., 0 to 1) L Original More Contrast

17 Image Processing Pixel operations Add random noise Add luminance Add contrast Add saturation Filtering Blur Detect edges Sharpen Emboss Median Quantization Uniform Quantization Floyd-Steinberg dither Warping Scale Rotate Warps Combining Composite Morph

18 Linear Filtering (Spatial Domain) Convolution Each output pixel is a linear combination of input pixels in neighborhood with weights prescribed by a filter Filter = 18

19 Adjust Blurriness Convolve with a filter whose entries sum to one Each pixel becomes a weighted average of its neighbors Original Blur What do you think happens in the frequency domain? Filter =

20 More on blur (lowpass filters) We can either take a uniform kernel (mean filter) Or a Gaussian kernel A Gaussian kernel tends to provide gentler smoothing and preserve edges better

21 Edge Detection Convolve with a filter that finds differences between neighbor pixels Original Detect edges 1 Filter =

22 Sharpen Sum detected edges with original image Original Sharpened Filter =

23 Emboss Convolve with a filter that highlights gradients in particular directions Original Embossed Filter =

24 Non-linear filtering Any operation on a neighborhood around each pixel For example: Selecting the median value of the neighborhood Original 3x3 5x5 7x7 11x11 15x15 24

25 Image Processing Pixel operations Add random noise Add luminance Add contrast Add saturation Filtering Blur Detect edges Sharpen Emboss Median Quantization Uniform Quantization Floyd-Steinberg dither Warping Scale Rotate Warps Combining Composite Morph

26 Reduce intensity resolution Quantization Frame buffers have limited number of bits per pixel Physical devices have limited dynamic range n=0.5 26

27 P(x,y) = round(i(x,y)) Uniform Quantization I(x,y) P(x,y) 2 bits per pixel 27

28 Uniform Quantization Images with decreasing bits per pixel: 8 bits 4 bits 2 bits 1 bit 28

29 Reducing effects of Quantization Dithering Random dither Ordered dither Error diffusion dither Halftoning Classical halftoning 29

30 Dithering Distribute errors among pixels Exploit spatial integration in our eye Display greater range of perceptible intensities Original (8 bits) Uniform Quantization (1 bit) Floyd-Steinberg Dither (1 bit)

31 P(x,y) P(x,y) Randomize quantization errors Errors appear as noise Random Dither I(x,y) I(x,y) P(x, y) = trunc(i(x, y) + noise(x,y) + 0.5) 1 bit

32 Random Dither Original (8 bits) Uniform Quantization (1 bit) Random Dither (1 bit)

33 Ordered Dither Pseudo-random quantization errors Matrix stores pattern of threshholds D 2 For each pixel (x,y) oldpixel = I(x,y) +D(x mod n,y mod n) P(x,y)= find_closest_color(oldpixel)

34 Ordered Dither Bayer s ordered dither matrices D D (2,2) 4 (2,1) 4 (1,2) 4 (1,1) 4 n n n n n n n n n U D D U D D U D D U D D D Basic idea: organize successive integers such that the average distance between two successive numbers in the map is as large as possible

35 An example Ordered Dither Palette consists of 8 red tones, 8 green tones and their combinations (64 colors) Original image had colors Undithered Dithered 35

36 Ordered Dither Original (8 bits) Random Dither (1 bit) Ordered Dither (1 bit)

37 Error Diffusion Dither Spread quantization error over neighbor pixels Error dispersed to pixels right and below a b g d a b g d 1.0 Figure from H&B

38 Floyd-Steinberg Algorithm for (x = 0; x < width; x++) { for (y = 0; y < height; y++) { P(x,y) = trunc(i(x,y) + 0.5) e = I(x,y) - P(x,y) I(x,y+1) += a*e; I(x+1,y-1) += b*e; I(x+1,y) += g*e; I(x+1,y+1) += d *e; } }

39 Error Diffusion Dither Original (8 bits) Random Dither (1 bit) Ordered Dither (1 bit) Floyd-Steinberg Dither (1 bit)

40 More examples Original Threshold Random Bayer Floyd-Steinberg Jarvice, Judice & Ninke Stucki Burkes 40

41 Reducing effects of Quantization Dithering Random dither Ordered dither Error diffusion dither Halftoning Classical halftoning 41

42 Classical Halftoning Use dots of varying size to represent intensities Area of dots proportional to intensity in image I(x,y) P(x,y)

43 Classical Halftoning Newspaper Image From New York Times, 9/21/99

44 Halftone patterns Use cluster of pixels to represent intensity Trade spatial resolution for intensity resolution Figure from H&B

45 Halftone patterns How many intensities in a n x n cluster? Figure from H&B

46 Image Processing Pixel operations Add random noise Add luminance Add contrast Add saturation Filtering Blur Detect edges Sharpen Emboss Median Quantization Uniform Quantization Floyd-Steinberg dither Warping Scale Rotate Warps Combining Composite Morph

47 Image Warping Move pixels of image Warp Source image Destination image

48 Image Warping Issues How do we specify where every pixel goes? (mapping) How do we compute colors at destination pixels? (resampling) Warp Source image Destination image

49 Example Image Scaling (x,y ) = (sx*x, sy*y); I(x,y ) =? 49

50 Image Warping Image warping requires resampling of image Resampling 50

51 BACK TO SAMPLING 51

52 Aliasing (again) In general: Artifacts due to under-sampling or poor reconstruction Specifically, in graphics: Spatial aliasing Temporal aliasing Under-sampling Figure FvDFH

53 Spatial Aliasing Artifacts due to limited spatial resolution

54 Spatial Aliasing Artifacts due to limited spatial resolution Jaggies

55 Temporal Aliasing Artifacts due to limited temporal resolution Strobing Flickering

56 Temporal Aliasing Artifacts due to limited temporal resolution Strobing Flickering

57 Temporal Aliasing Artifacts due to limited temporal resolution Strobing Flickering

58 Temporal Aliasing Artifacts due to limited temporal resolution Strobing Flickering

59 Sample at higher rate Not always possible Doesn t always solve problem Antialiasing Pre-filter to form bandlimited signal Form bandlimited function (low-pass filter) Trades aliasing for blurring

60 Image Processing Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display

61 Image Processing Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display Continuous Function

62 Image Processing Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display Discrete Samples

63 Image Processing Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display Reconstructed Function

64 Image Processing Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display Transformed Function

65 Image Processing Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display Bandlimited Function

66 Image Processing Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display Discrete samples

67 Image Processing Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display Display

68 Frequency domain Ideal Bandlimiting Filter Spatial domain Sinc( x) sin x x Figure 4.5 Wolberg

69 Convolution Practical Image Processing Finite low-pass filters Point sampling (bad) Triangle filter Gaussian filter Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display

70 Convolution with triangle filter Triangle Filter Input Output Figure 2.4 Wolberg

71 Gaussian Filter Convolution with Gaussian filter Input Output Figure 2.4 Wolberg

72 AND BACK TO WARPING 72

73 Image Resampling What if we are resampling a 2D image? (u,v)

74 Image Resampling Compute weighted sum of pixel neighborhood Output is weighted average dst(u,v)=0; for(ix=u-w;ix<=u+w;ix++) for(iy=v-w;iy<=v+w;iy++) d=dist between (ix,iy) and (u,v) dst(u,v) += k(ix,iy) * src(ix,iy) (u,v) W d (ix,iy)

75 Image Resampling For isotropic Triangle and Gaussian filters, k(ix,iy) is a function of d and w (u,v) W d (ix,iy)

76 Image Resampling For isotropic Triangle and Gaussian filters, k(ix,iy) is a function of d and w (u,v) W d (ix,iy)

77 Triangle Filtering (width <= 1) Bilinearly interpolate four closest pixels a = linear interpolation of src(u 1,v 2 ) and src(u 2,v 2 ) b = linear interpolation of src(u 1,v 1 ) and src(u 2,v 1 ) dst(x,y) = linear interpolation of a and b (u 1,v 2 ) a (u 2,v 2 ) (u,v) (u 1,v 1 ) b (u 2,v 1 )

78 Kernel is a Guassian function Gaussian Filtering (u,v) w 3 d (ix,iy)

79 Image Scale Scale (src, dst, sx, sy): w max(1/sx,1/sy); for (int ix = 0; ix < xmax; ix++) { for (int iy = 0; iy < ymax; iy++) { float u = ix / sx; float v = iy / sy; dst(ix,iy) = resample(src,u,v,k,w); } } v (u,v) y (x,y) u Scale 0.5 x

80 How do we resample? Point sampling Simple but causes aliasing Triangle and Gaussian Algorithm as we saw earlier Float resample(src,u,v,w) { int iu = round(u); int iv = round(v); return src(iu,iv); } 80

81 Image Warping (in General) Reverse Mapping 81

82 Image Warping (in General) Alternative (forward) 82

83 Next time? That s it for today Finishing corners on image processing Transformations and Projections Rendering 83